Methodology for automatic classification of atypical lymphoid cells from peripheral blood cell images

Morphological analysis is the starting point for the diagnostic approach of more than 80% of the hematological diseases. However, the morphological differentiation among different types of abnormal lymphoid cells in peripheral blood is a difficult task, which requires high experience and skill. Objective values do not exist to define cytological variables, which sometimes results in doubts on the correct cell classification in the daily hospital routine. Automated systems exist which are able to get an automatic preclassification of the normal blood cells, but fail in the automatic recognition of the abnormal lymphoid cells. The general objective of this thesis is to develop a complete methodology to automatically recognize images of normal and reactive lymphocytes, and several types of neoplastic lymphoid cells circulating in peripheral blood in some mature B-cell neoplasms using digital image processing methods. This objective follows two directions: (1) with engineering and mathematical background, transversal methodologies and software tools are developed; and (2) with a view towards the clinical laboratory diagnosis, a system prototype is built and validated, whose input is a set of pathological cell images from individual patients, and whose output is the automatic classification in one of the groups of the different pathologies included in the system. This thesis is the evolution of various works, starting with a discrimination between normal lymphocytes and two types of neoplastic lymphoid cells, and ending with the design of a system for the automatic recognition of normal lymphocytes and five types of neoplastic lymphoid cells. All this work involves the development of a robust segmentation methodology using color clustering, which is able to separate three regions of interest: cell, nucleus and peripheral zone around the cell. A complete lymphoid cell description is developed by extracting features related to size, shape, texture and color. To reduce the complexity of the process, a feature selection is performed using information theory. Then, several classifiers are implemented to automatically recognize different types of lymphoid cells. The best classification results are achieved using support vector machines with radial basis function kernel. The methodology developed, which combines medical, engineering and mathematical backgrounds, is the first step to design a practical hematological diagnosis support tool in the near future.Los análisis morfológicos son el punto de partida para la orientación diagnóstica en más del 80% de las enfermedades hematológicas. Sin embargo, la clasificación morfológica entre diferentes tipos de células linfoides anormales en la sangre es una tarea difícil que requiere gran experiencia y habilidad. No existen valores objetivos para definir variables citológicas, lo que en ocasiones genera dudas en la correcta clasificación de las células en la práctica diaria en un laboratorio clínico. Existen sistemas automáticos que realizan una preclasificación automática de las células sanguíneas, pero no son capaces de diferenciar automáticamente las células linfoides anormales. El objetivo general de esta tesis es el desarrollo de una metodología completa para el reconocimiento automático de imágenes de linfocitos normales y reactivos, y de varios tipos de células linfoides neoplásicas circulantes en sangre periférica en algunos tipos de neoplasias linfoides B maduras, usando métodos de procesamiento digital de imágenes. Este objetivo sigue dos direcciones: (1) con una orientación propia de la ingeniería y la matemática de soporte, se desarrollan las metodologías transversales y las herramientas de software para su implementación; y (2) con un enfoque orientado al diagnóstico desde el laboratorio clínico, se construye y se valida un prototipo de un sistema cuya entrada es un conjunto de imágenes de células patológicas de pacientes analizados de forma individual, obtenidas mediante microscopía y cámara digital, y cuya salida es la clasificación automática en uno de los grupos de las distintas patologías incluidas en el sistema. Esta tesis es el resultado de la evolución de varios trabajos, comenzando con una discriminación entre linfocitos normales y dos tipos de células linfoides neoplásicas, y terminando con el diseño de un sistema para el reconocimiento automático de linfocitos normales y reactivos, y cinco tipos de células linfoides neoplásicas. Todo este trabajo involucra el desarrollo de una metodología de segmentación robusta usando agrupamiento por color, la cual es capaz de separar tres regiones de interés: la célula, el núcleo y la zona externa alrededor de la célula. Se desarrolla una descripción completa de la célula linfoide mediante la extracción de descriptores relacionados con el tamaño, la forma, la textura y el color. Para reducir la complejidad del proceso, se realiza una selección de descriptores usando teoría de la información. Posteriormente, se implementan varios clasificadores para reconocer automáticamente diferentes tipos de células linfoides. Los mejores resultados de clasificación se logran utilizando máquinas de soporte vectorial con núcleo de base radial. La metodología desarrollada, que combina conocimientos médicos, matemáticos y de ingeniería, es el primer paso para el diseño de una herramienta práctica de soporte al diagnóstico hematológico en un futuro cercano

Characterization and automatic screening of reactive and abnormal neoplastic B lymphoid cells from peripheral blood

The objective was to advance in the automatic, image-based, characterization and recognition of a heterogeneous set of lymphoid cells from peripheral blood, including normal, reactive, and five groups of abnormal lymphocytes: hairy cells, mantle cells, follicular lymphoma, chronic lymphocytic leukemia, and prolymphocytes. Methods: A number of 4389 images from 105 patients were selected by pathologists, based on morphologic visual appearance, from patients whose diagnosis was confirmed by all the remaining complementary tests. Besides geometry, new color and texture features were extracted using six alternative color spaces to obtain rich information to characterize the cell groups. The recognition system was designed using support vector machines trained with the whole image set. Results: In the experimental tests, individual sets of images from 21 new patients were analyzed by the trained recognition system and compared with the true diagnosis. An overall recognition accuracy of 97.67% was achieved when the cell screening was performed into three groups: normal lymphocytes, abnormal lymphoid cells, and reactive lymphocytes. The accuracy of the whole experimental study was 91.23% when considering the further discrimination of the abnormal lymphoid cells into the specific five groups. Conclusion: The excellent automatic screening of the three groups of normal, reactive, and abnormal lymphocytes is useful as it discriminates between malignancy and not malignancy. The discrimination of the five groups of abnormal lymphoid cells is encouraging toward the idea that the system could be an automated image-based screening method to identify blood involvement by a variety of B lymphomas.Preprin

Optimizing morphology through blood cell image analysis

Introduction Morphological review of the peripheral blood smear is still a crucial diagnostic aid as it provides relevant information related to the diagnosis and is important for selection of additional techniques. Nevertheless, the distinctive cytological characteristics of the blood cells are subjective and influenced by the reviewer's interpretation and, because of that, translating subjective morphological examination into objective parameters is a challenge. Methods The use of digital microscopy systems has been extended in the clinical laboratories. As automatic analyzers have some limitations for abnormal or neoplastic cell detection, it is interesting to identify quantitative features through digital image analysis for morphological characteristics of different cells. Result Three main classes of features are used as follows: geometric, color, and texture. Geometric parameters (nucleus/cytoplasmic ratio, cellular area, nucleus perimeter, cytoplasmic profile, RBC proximity, and others) are familiar to pathologists, as they are related to the visual cell patterns. Different color spaces can be used to investigate the rich amount of information that color may offer to describe abnormal lymphoid or blast cells. Texture is related to spatial patterns of color or intensities, which can be visually detected and quantitatively represented using statistical tools. Conclusion This study reviews current and new quantitative features, which can contribute to optimize morphology through blood cell digital image processing techniques.Peer ReviewedPostprint (published version

Image processing and machine learning in the morphological analysis of blood cells

Introduction: This review focuses on how image processing and machine learning can be useful for the morphological characterization and automatic recognition of cell images captured from peripheral blood smears. Methods: The basics of the 3 core elements (segmentation, quantitative features, and classification) are outlined, and recent literature is discussed. Although red blood cells are a significant part of this context, this study focuses on malignant lymphoid cells and blast cells. Results: There is no doubt that these technologies may help the cytologist to perform efficient, objective, and fast morphological analysis of blood cells. They may also help in the interpretation of some morphological features and may serve as learning and survey tools. Conclusion: Although research is still needed, it is important to define screening strategies to exploit the potential of image-based automatic recognition systems integrated in the daily routine of laboratories along with other analysis methodologies.Peer ReviewedPostprint (published version

Automatic normalized digital color staining in the recognition of abnormal blood cells using generative adversarial networks

Background and Objectives: Combining knowledge of clinical pathologists and deep learning models is a growing trend in morphological analysis of cells circulating in blood to add objectivity, accuracy, and speed in diagnosing hematological and non-hematological diseases. However, the variability in staining protocols across different laboratories can affect the color of images and performance of automatic recognition models. The objective of this work is to develop, train and evaluate a new system for the normalization of color staining of peripheral blood cell images, so that it transforms images from different centers to map the color staining of a reference center (RC) while preserving the structural morphological features. Methods: The system has two modules, GAN1 and GAN2. GAN1 uses the PIX2PIX technique to fade original color images to an adaptive gray, while GAN2 transforms them into RGB normalized images. Both GANs have a similar structure, where the generator is a U-NET convolutional neural network with ResNet and the discriminator is a classifier with ResNet34 structure. Digitally stained images were evaluated using GAN metrics and histograms to assess the ability to modify color without altering cell morphology. The system was also evaluated as a pre-processing tool before cells undergo a classification process. For this purpose, a CNN classifier was designed for three classes: abnormal lymphocytes, blasts and reactive lymphocytes. Results: Training of all GANs and the classifier was performed using RC images, while evaluations were conducted using images from four other centers. Classification tests were performed before and after applying the stain normalization system. The overall accuracy reached a similar value around 96% in both cases for the RC images, indicating the neutrality of the normalization model for the reference images. On the contrary, it was a significant improvement in the classification performance when applying the stain normalization to the other centers. Reactive lymphocytes were the most sensitive to stain normalization, with true positive rates (TPR) increasing from 46.3% - 66% for the original images to 81.2% - 97.2% after digital staining. Abnormal lymphocytes TPR ranged from 31.9% - 95.7% with original images to 83% - 100% with digitally stained images. Blast class showed TPR ranges of 90.3% - 94.4% and 94.4% - 100%, for original and stained images, respectively. Conclusions: The proposed GAN-based normalization staining approach improves the performance of classifiers with multicenter data sets by generating digitally stained images with a quality similar to the original images and adaptability to a reference staining standard. The system requires low computation cost and can help improve the performance of automatic recognition models in clinical settings.This work is part of a research project funded by the Ministry of Science and Innovation of Spain, with reference PID2019-104087RB-I00.Peer ReviewedPostprint (published version

Multidimensional big data processing for damage detection in real pipelines using a smart pig tool

The history of the hydrocarbons business in Colombia dates back to the early twentieth century where mining and energy sector has been one of the principal pillars for the its development. Thus, the pipelines currently in service have over 30 years and most of them are buried and phenomena like metal losses, corrosion, mechanical stress, strike by excavation machinery and other type of damages are presented. Since it can generate social and environmental problems, monitoring tools and programs should be developed in order to prevent catastrophic situations. However, the maintaining of these structures is very expensive and it is normally developed by foreign companies. In order to overcome this situation, recently the native research institute “Research Institute of Corrosion - CIC (Corporación para la Investigación de la Corrosión)” developed an in-line inspection tool to be operated in Colombian pipelines (especially gas) to get valuable information of their current state along of thousand kilometres. The recorded data is of big size and its processing demand a high computational cost and adequate tool analysis to determine a certain pipeline damage condition. On other hand, the author from UPC and UIS have been bringing its expertise in processing and analysing this type of big data by using mainly Principal Component Analysis (PCA) as an effective tool to detect and locate different damages. In previous papers, multidimensional data matrix was used to locate possible damages along the pipeline, however most of activated points were considered false alarms since they corresponded to weld points. Thus, in this paper it is proposed no considering piecewise weld points (tube sections) and an extension of PCA named Multiway PCA (MPCA) is applied for each each one of the tube sections that form the pipeline. Therefore, if a tube section is found outside from overall indices found by using the MPCA model, an alarm activated in that section and a precise location can be obtained by analyzing only data from that specific tube section.Postprint (author's final draft

Digital blood image processing and fuzzy clustering for detection and classification of atypical lymphoid B cells

Automated systems for digital peripheral blood (PB) cell analysis operate most effectively in non-pathological samples. The paper deals with the automatic classification of atypical lymphoid cells using digital image processing. The problem has been approached through a 3-step procedure: 1) Watershed segmentation of nucleus, cytoplasm and peripheral cell zone; 2) feature extraction for each region; and 3) classification using fuzzy c-means. The paper has proposed a new methodology that has been able to automatically classify with high precision three types of lymphoid cells: normal, Hairy Cell Leukemia cells and Chronic Lymphocytic Leukemia cells. This methodology, combining human medical expertise with mathematical and engineering tools, may contribute to improve the efficiency of the hematology laboratory.Peer Reviewe

A dataset of microscopic peripheral blood cell images for development of automatic recognition systems

This article makes available a dataset that was used for the development of an automatic recognition system of peripheral blood cell images using convolutional neural networks [1]. The dataset contains a total of 17,092 images of individual normal cells, which were acquired using the analyzer CellaVision DM96 in the Core Laboratory at the Hospital Clinic of Barcelona. The dataset is organized in the following eight groups: neutrophils, eosinophils, basophils, lymphocytes, monocytes, immature granulocytes (promyelocytes, myelocytes, and metamyelocytes), erythroblasts and platelets or thrombocytes. The size of the images is 360¿×¿363 pixels, in format jpg, and they were annotated by expert clinical pathologists. The images were captured from individuals without infection, hematologic or oncologic disease and free of any pharmacologic treatment at the moment of blood collection. This high-quality labelled dataset may be used to train and test machine learning and deep learning models to recognize different types of normal peripheral blood cells. To our knowledge, this is the first publicly available set with large numbers of normal peripheral blood cells, so that it is expected to be a canonical dataset for model benchmarking.Peer ReviewedPostprint (published version

A deep learning pproach for the morphological recognition of reactive lymphocytes in patients with COVID-19 infection

Laboratory medicine plays a fundamental role in the detection, diagnosis and management of COVID-19 infection. Recent observations of the morphology of cells circulating in blood found the presence of particular reactive lymphocytes (COVID-19 RL) in some of the infected patients and demonstrated that it was an indicator of a better prognosis of the disease. Visual morphological analysis is time consuming, requires smear review by expert clinical pathologists, and is prone to subjectivity. This paper presents a convolutional neural network system designed for automatic recognition of COVID-19 RL. It is based on the Xception71 structure and is trained using images of blood cells from real infected patients. An experimental study is carried out with a group of 92 individuals. The input for the system is a set of images selected by the clinical pathologist from the blood smear of a patient. The output is the prediction whether the patient belongs to the group associated with better prognosis of the disease. A threshold is obtained for the classification system to predict that the smear belongs to this group. With this threshold, the experimental test shows excellent performance metrics: 98.3% sensitivity and precision, 97.1% specificity, and 97.8% accuracy. The system does not require costly calculations and can potentially be integrated into clinical practice to assist clinical pathologists in a more objective smear review for early prognosis.Peer ReviewedPostprint (published version

Texture analysis for wind turbine fault detection

The future of wind energy industry passes through the use of larger and more flexible wind turbines in remote locations, which are increasingly offshore to benefit stronger and more uniform wind conditions. Cost of operation and maintenance of offshore wind turbines is among 15-35% of the total cost. From this, 80% comes from unplanned maintenance due to different faults in the wind turbine components. Thus, an auspicious way to contribute to the increasing demands and challenges is by applying low-cost advanced fault detection schemes. This work proposes a new method for fault detection of wind turbine actuators and sensors faults in variable-speed wind turbines. For this purpose, time domain signals acquired from the operating wind turbine are converted into two-dimensional matrices to obtain gray-scale digital images. Then, the image pattern recognition is processed getting texture features under a multichannel representation. In this work, four types of texture features are used: statistical, wavelet, granulometric and Gabor features. Then, the most significant features are selected with the conditional mutual criterion. Finally, the fault detection is performed using an automatic classification tool. In particular, a 10-fold cross validation is used to obtain a more generalized model and evaluate the classification performance. In this way, the healthy and faulty conditions of the wind turbine can be detected. Coupled non-linear aero-hydro-servo-elastic simulations of a 5MW offshore type wind turbine are carried out for several fault scenarios. The results show a promising methodology able to detect the most common wind turbine faults.Postprint (published version